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EXECUTIVE SUMMARY 


Introduction 

This study contributes knowledge about how a digital badge system 
integrated into an online, subject-matter-specific, and competency-based 
professional development (PD) program affected teachers' experiences 
with and perceptions of the program activities. 

The report presents findings from a one-year exploratory study of an 
online PD program, and accompanying digital badge system, called Who 
Built America Badges: Common Core Professional Development from the American 
Social History Project. The American Social History Project (AS HP), a well- 
established history and social studies professional development provider 
in New York City, created the system to make its PD program available to a 
national audience of middle and high school teachers. The program's main 
goal is to help teachers develop new instructional routines that build 
students' reading and writing practices that are specific to the domain of 
history, as well as their historical reasoning skills. 

What is unique to the WB A PD program — and significant to the broader 
enterprise of teacher professional development — is that each digital badge signifies 
an instance of PD activity that is linked to a larger, history education-focused 
system that builds teachers’ mastery of content and instructional practices. As 
conceived, the badge system was intended to enable incremental, cohesive 
approaches to inservice professional development. Some badges are 
obtainable after a single activity, while others are “prerequisites” that 
constitute a more advanced badge when taken collectively. But each 
represents an element of a comprehensive concept of “mastery,” as defined 
by the teacher educators and social historians at AS HP. 

Many questions about badges remain to be investigated in future 
studies, starting with whether badge systems can be accepted among 
professional educators as possible solutions to some of the longstanding 
shortcomings in teacher PD. Digital badge systems like this one (i.e., that 
include research-based features associated with effective PD), however, 
could enable professional learning programs to credential inservice 
teacher development systematically and over time, supporting gradual 
improvements to teaching practice in a specific subject area. 
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The WBA program accommodates teachers' immediate practical needs 
for classroom-ready materials, while promoting long-term development of 
instructional practices that align to AS HP's vision of a master history 
teacher's practices. Whether that vision accords with the priorities of 
teachers and schools depends on contextual factors well beyond the scope 
of this study, but knowing will be critical to determining whether badges 
become valued, or even acceptable, as PD credentials. 

Implications of the study 

A frequent criticism of inservice teacher professional development is that 
it is removed from genuine practices of teaching disciplinary knowledge to 
diverse groups of learners. It is unlikely that many programs can be 
legitimate sites for advancing teachers' pedagogical content knowledge 
when inservice PD exists primarily as standalone "workshops,” rather than 
as ongoing programs of professional learning that enable teachers to grow 
through and with practice in their specific subjects. This deficiency is 
especially troubling when it coexists with the types of large-scale 
curriculum and state-level assessment realignments that occur due to the 
adoption of standards such as the Common Core. The scope of those 
changes and the level of expectations for student learning outcomes 
ensure that one or two day workshops that address "generic teaching 
practices” will not always suffice. 

A digital badge system like WBA presents the possibility of offering a 
sustained program of online professional learning for history and social 
studies educators that signals teacher achievements in their area of 
specialization. Badges as P D credentials are essentially unheard of in every 
state, however, and the format of ongoing, competency-based professional 
development administered by external providers is not common. Not a 
single teacher whom we interviewed as part of this study received 
continuing education credit for their participation. 

As this report should make clear, the innovation is not badges, but 
rather a badge system that supports and signals PD activities that many 
teachers, teacher educators, and historians value. Most teachers whom we 
interviewed valued the materials and the activities and intend to return to 
the site, perhaps even to work toward earning badges. Without external 
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authorities such as school or district administrators to recognize the 
badges, however, it will be very difficult to sustain a program that 
demands so much of teachers, given other demands on their time. Once 
AS HP secures commitments from external agencies to support and 
recognize this type of PD, it should consider a second, larger study to 
investigate the impact of the badge system on teacher and student 
learning outcomes. 

Research questions 

Our main question was, "In what ways do the features of a digital badge system 
support or impede history and social studies teachers’ progress toward mastery of a 
new set of practices in a competency-based professional development program?” 
Two specific questions guided the data collection and analysis: 

1. What role does working toward and achieving WBA badges play in the 
following aspects of social studies teachers’ professional development: 
continuing participation in the project; peer perceptions of a teacher’s 
professional capabilities; school and district administrators’ willingness to 
treat badges as professional credentials; and teachers’ interests in 
developing new shills? 

2 . In what ways do badge-related activities ( including unit design, peer 
collaboration, and assessment by master teachers) influence the 
development of an online community of practice? 

Methods 

The study included 29 middle and high school history and social studies 
teachers in 13 states between September 2013 and September 2014. During 
the 2013-2014 school year, participating teachers agreed to download at 
least one of four “inquiry units” from the WBA web site; modify existing 
lesson plans in the unit; teach them; share reflections with teacher 
educators; and respond to comments and critiques from their peers and 
teacher educators. As they completed tasks, teachers received a badge from 
one of three categories, depending on the nature of the activity. 

Data collection. We collected data between November 2013 and October 
2014. Data included: (1) Baseline "literacy-focused” lesson plans that pre- 
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dated the study; (2) System-generated logs for all teachers who earned at 
least one badge beyond the WBA Member (users receive these automatically 
after registering); (3) Pre-participation surveys; (4) Surveys following 
badge awarding (6 surveys max.); (5) Communication logs from the teacher 
forum; ( 6 ) Communications with AS H P teacher educators via the 
submission forms; and (7) Follow-up phone interviews with nine teachers. 

Data analysis. We used teachers' pre-participation surveys to describe 
their previous experiences with “generic” PD and Common Core-related 
PD; teaching Common Core-aligned lessons; and previous experience with 
badges/digital badges. The system-generated logs enabled us to count the 
types of badges teachers earned. To analyze teachers' comments from the 
badge surveys and follow-up interviews, we conducted a thematic analysis 
using the analysis framework summarized below. We also created a small 
set of “inductive” codes as we reviewed the data and learned more about 
teachers' perceptions of the badges. 

Analytic framework. We coded teachers' interview comments and survey 
responses using an initial set of codes based on evidence-based "core 
features” of effective professional development. 


PD feature 

Description 

1. Content focus 

A sustained focus on a teacher’s subject area, connected to standards, 
curriculum, instruction, assessment, and knowledge of how students 
learn in that content area 

2. Active 
learning 

Teachers should be actively involved in the PD activities, engaged in 
activities such as looking at student work, receiving feedback on 
teaching, giving feedback to peers, or participating in lesson studies 

3. Duration 

PD activities should be sustained over time and focused on content, 
curriculum, and student activities 

4. Collective 
participation 

Teachers from the same grade level, subject area, or school should 

engage in PD activities together 

5. Coherence 

PD activities should be consistent with other professional 
development, existing knowledge and beliefs, and with school, district, 
and state reforms and policies 
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Key findings 

• 356 educators from 41 states and the District of Columbia registered 
for the WBA site between November 2013 and October 2014 

• 29 teachers from 13 states participated in the study: 11 from NY; five 
from AZ; two each from I<Y and IA; and one each from CA, IL, IN, 
MN, MO, NH, UT, VA, and WA 

• A very small number of teachers achieved badges during the study: 

2 of 29 study participants (7%) achieved an Apprentice Builder 
badge, a core badge related to the program's PD objectives 

• Similarly, a very small number of registered users achieved an 
Apprentice Builder badge: 5 of 327 (1.5%) 

• Teachers were drawn to the WBA project primarily for access to 
what they perceived to be very high-quality teaching materials; only 
2 of the 9 teachers whom we interviewed indicated an interest in 
badges as motivating them to explore the WBA site 

• None of the teachers will receive continuing education credits for 
the work they did as part of the project 

• 7 of the 9 teachers whom we interviewed commented that the 
ASHP-produced inquiry units effectively modeled practices that 
they would like to continue to develop and integrate into their 
routines over time. Each of these teachers said that they felt this 
type of work was important for students and teachers as a part of 
history and social studies education 

• The WBA program placed significant demands on participants' 
time and energy: 7 of the 9 teachers said that they would be willing 
to continue to pursue this form of professional development, but 
only if there were changes to the pacing, the amount of material 
they needed to submit, and the quality and frequency of the 
feedback from AS HP 

• Of the five features in the analytic framework, teachers’ survey 
responses and interview comments suggest that the PD activities 
and the badges evidenced aspects of all five features, but to 
different degrees of success. Specifically: 

o Content focus: For the majority of participants, the activities 
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and the badges aligned to their content and pedagogical 
needs and interests and were the most influential factor in 
their decision to participate in the PD program. But most 
teachers referred to the quality of the historical content and 
the source documents, rather than the focus on Common 
Core-aligned skills. Additionally, all nine of the interviewees 
remarked that there was not enough content on the site for 
them to remain engaged over the school year. 

o Active learning: The activities and badges are designed to 
support to active participation among teachers. Most of the 
teachers felt that the level of participation was too high, 
however. That is, earning badges generally required too much 
effort in terms of the amount of work they had to submit and 
the activities they had to document. 

o Duration: The activities and the badges are designed to 
encourage iterative changes to teaching materials and 
practices over time. The majority of teachers said that they 
would be willing to persist over time, but for them to do so 
would require less work and more effective communication 
with peers and the teacher educators. 

o Collective participation: Teachers generally responded 

negatively to the level of collective participation. While the 
badge system and teacher forum were designed to encourage 
collaboration, most teachers felt that the forum was poorly 
designed and that AS H P had not done enough to promote 
collaboration among teachers. 

o Coherence: The majority of teachers thought that the PD 

activities and badges were “internally coherent” — that is, the 
progression logic of the badge system made sense and 
generally reflected their own understanding of what it could 
mean to become a master history teacher. Conversely, all 
teachers felt that the badge system was not coherent with 
local policies and priorities. None of the teachers with whom 
we spoke had principals or districts that encouraged 
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competency-based professional development, nor did they 
recognize digital badges as a form of credentialing. Further, 
because districts and states have different priorities with 
respect to the Common Core, the WBA program's focus on the 
standards was not necessarily compelling. 

Recommendations for improving the WBA badge system 

• Create “onboarding” badges to decrease the amount of time it takes 
to earn a "meaningful” badge 

• Identify local AND state- or district-level partners to grant credit 
for badges (or provide teachers with other opportunities, such as 
release time, to participate) 

• Improve mechanisms to support asynchronous and synchronous 
communications among teachers and teacher educators 

• Increase the frequency of communication between the teacher 
educators and registered users 

• Improve the web site layout and print material layout in order to 
make the program more approachable and useful for all users 

Recommendations for future research 

It is too soon to investigate questions about whether badge systems might 
be effective tools for promoting teacher professional development. Our 
experience during this study suggests that, before researchers focus on the 
qualities of any given badge system, we must first address questions about 
the broader sociotechnical contexts in which the badge systems will exist. 
Accordingly, we should investigate whether individual teachers and their 
schools, districts, and communities value the activities and objectives of 
any given badge system. Future work should use anthropological and 
sociological methods to investigate the sociocultural challenges associated 
with adopting this kind of innovation, as well as to identify the existing 
social and technical structures that will influence whether the innovation 
can be successful. 
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I. INTRODUCTION AND STUDY CONTEXT 


Introduction 

In March 2012, the American Social History Project (AS HP), a K-12 history 
and social studies teacher professional development (PD) organization and 
social history research center at the Graduate Center of the City University 
of New York (CUNY), submitted a proposal to HASTAC’s Badges for 
Lifelong Learning Competition. AS HP is a well-established teacher PD 
provider to the New York City public school system. The organization was 
the sole recipient of an award in the “Teacher Mastery and Feedback 
Badge” category for its plan to create a unique, and free, online history and 
social studies teacher professional learning program called Who Built 
America Badges: Common Core Professional Development from the American 
Social Histoiy Project ( WBA). T The PD program would use digital badges to 
recognize teachers' achievements during professional development. 

The objective for the competition's Teacher Mastery category was to 
support the development of a badging system to recognize online, 
inservice teacher professional learning activities using a competency- 
based approach to professional education. That approach contrasts with 
the traditional “seat time” model that grants continuing education units 
based solely on total hours of participation. The WBA team, led by teacher 
educators fromASHP and including two NYC -based development 
partners (Electric Funstuff and ThinkDesign^) and a learning assessment 
partner (EDC), received a development grant from the Bill and Melinda 
Gates Foundation to build and pilot the system. Development started in 
late 2012. 

The 2012 Digital Media and Learning (DML) Competition sought to 
stimulate the creation of digital badging — or “micro-credential” — systems 
that recognized “anytime, anywhere” learning for youth and adults. 


* Humanities, Arts, Science and Technology Alliance and Collaboratory (http://www.hastac.org) 
1 The name Who Built America is taken from ASHP’s American history textbook of the same title 
1 http://dmlhub.net/newsroom/media-releases/hastac-announces-new-teacher-mastery-and- 
feedback-badge-competition 

§ http://electricfunstuff.com; http://www.thinkdesign.com; a third development partner, 
SOSBrooldyn (http://sosbrooklyn.com), replaced ThinkDesign in summer 2013 
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Fostering the development of these systems was part of an ongoing 
initiative, funded by the MacArthur and Mozilla Foundations, to promote 
educational equity by expanding opportunities for learners to discover and 
build personalized college and career trajectories. By encouraging the 
spread of technologies that might enable learners to document, share, and 
receive recognition for personally meaningful activities that often go 
unrecognized by formal credentialing systems (but that might otherwise 
be valuable), the initiative's goal is to support “connected learning," a 
framework for “deploying new media to reach and enable youth who 
otherwise lack access to opportunity... [by building] communities and 
collective capacities for learning and opportunity" (Ito, et ah, 2013, p. 8 ). 

The Teacher Mastery category of the DML competition extended 
connected learning principles to teacher professional development. The 
competition supported the creation of an open-source technical 
infrastructure that enabled teachers to pursue personalized learning 
opportunities and display their achievements from those activities in 
publicly accessible digital spaces. 

Following the initial badge development competition, HASTAC 
administered a Research Competition for Badging and Badge Systems 
Development later in 2012. With one of five winning proposals. Education 
Development Center, Inc.'s Center for Children and Technology received a 
research grant from the Bill and Melinda Gates Foundation. The grant was 
to lead a study about the efficacy of using the WBA badge system to 
support history and social studies teachers' PD goals. 

EDC was the learning assessment partner on the preceding 
development grant and a long-time evaluation partner on AS HP's 
Teaching American History program-funded PD seminars. Hence, we 
were well-positioned to study the development process, design features, 
and “intended curriculum” (Porter & Smithson, 2001) of the badge system, 
as well as whether and how teachers who participated in the project used 
the badges to advance their professional development goals. Our report 
presents findings from a one-year exploratory study of the online PD 
program and WBA badge system. The study included 29 middle and high 


http://teachingamericanhistory.org/past-programs/tah/ 
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school history and social studies teachers in 13 states between September 
2013 and September 2014. The main research question was, “In what ways, if 
any, do the features of a digital badge system support or impede history and social 
studies teachers’ progress toward mastery of a new set of practices in a competency- 
based professional development program?” 

During the 2013-2014 school year, participants agreed to download at 
least one of four American history "inquiry units” (each from a different 
historical era) from the WBA web site;" modify existing lesson plans in the 
unit; teach them; submit annotated student work (teachers were required 
to submit work from a lesson that included a writing product); share 
reflections with teacher educators; and respond to comments and critiques 
from their peers and teacher educators. As they completed these tasks, 
teachers received one of 12 badges from three categories (Builder, 
Specialist, and Community), depending on the nature of the activity (we 
describe the badge system in Section II, p. 20). 

Following the data analysis, we discuss the relationship between the 
badge system and the format of these professional learning activities in the 
broader context of teacher professional development research and 
education reform. We also make recommendations for design features 
that might sustain and help scale-up the WBA badge system. Finally, we 
identify relevant factors for study in future investigations of this and other 
content-specific professional development badge systems. 

Importance of the study 

The adoption of the Common Core State Standards by 46 states and the 
District of Columbia since 2010 extends a near-continuous three decades 
of national education reform in the United States, with a narrowing focus 
on improving standards for learning and teaching. As reforms have 
proceeded at the national and local levels, advocates have regularly sought 
to improve teacher education and professional development programs in 
order to prepare teachers to address progressively ambitious student 
learning objectives (Wilson & Berne, 1999 ). In addition to other state 
mandates and local instructional expectations to which teachers might be 


" http://badges.ashp.cuny.edu 


EDC|Center for Children & Technology 
4 


held accountable, the Common Core introduced an emphasis on literacy 
instruction across all areas of the curriculum, necessitating new 
competencies beyond many teachers’ traditional content orientation. 

The standards also required “shifts” in teachers’ instructional practices 
to help students achieve the new learning objectives and to prepare them 
for the resultant state-level assessments (Student Achievement Partners, 
2014). In New York City, for example, secondary science and social studies 
teachers were required to adapt content-area units to the Common Core to 
ensure students had opportunities to build reading and writing skills 
across the disciplines (NYC DOE, 2012). The standards did not exist when 
many of these teachers completed their preservice training, necessitating 
inservice PD to help them integrate a focus on the reading and writing 
skills into their existing routines. 

Historically, a gap has existed between the types of expectations for 
higher-order student learning goals articulated in reform agendas such as 
the Common Core and the comprehensiveness of teacher professional 
development systems to help teachers and schools reach them. That gap 
has also extended to the allocation of resources to build those systems 
(Little, 1994). Significant challenges at the state policy and local 
implementation levels frequently prevent the kinds of systemic approaches 
to high-quality professional development that are necessary to reorient 
teaching and learning toward ambitious reform goals (Darling-Hammond, 
Wei, Richardson, Andree, & Orphanos, 2009; Knapp, 2003). 

During an earlier phase of education reform, Deborah Meier (1992) 
argued that the new types of expertise required of teachers demanded not 
simply more training, but rather, “creating a different system to do a 
different job... [but] the will to thoroughly restructure the institutions 
themselves, with all this implies. ..for massively retraining the educational 
work force, has little public backing” (p. 598). Effective teacher 
development programs are essential to the successful implementation of 
education standards (Fishman, Marx, Best, & Revital, 2003; Smylie, 1997 ). 
There are few examples of successful systemic teacher PD reforms that are 
tied to standards implementation, however. 

Well-designed online professional development programs can be 
valuable additions to “teacher learning communities” (Little, 2006) that 
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seek to improve teacher professional learning. The affordances of Web- 
based teacher PD programs — notably, the ability to offer a range of 
professional development services that teachers can access according to 
their own schedules, over time, and at a distance — generate opportunities 
to provide teachers with otherwise difficult-to-obtain high-quality PD 
materials repeatedly and as they need them (Dede, Ketelhut, Whitehouse, 
Breit, & McCloskey, 2009; Fishman, et al, 2013). 

The specific features of online professional development programs 
that contribute to improvements in teacher learning and practice are 
understudied, however, particularly in the areas of history and social 
studies. Moon, Passmore, Reiser, and Michaels (2013) argued that, “the 
field needs to go beyond treating modality as a main effect that considers 
online and face-to-face as two discrete forms of P D, and needs to 
investigate how these conditions interact with design features of the PD" 
(p. 173). This study responds to that challenge hy contributing findings about 
whether and how a novel feature of online PD — namely, a digital badge system — 
affected teachers’ abilities to benefit from other affordances of an online professional 
development program. 

We report on a qualitative study of a hybrid, online history teacher 
credentialing system that includes evidence-based features of PD that have 
been associated with improvements in teacher practice. The WBA badge 
system integrates (in a virtual format) elements of traditional, face-to-face, 
inservice professional development programs with a credentialing process 
that emulates standards-based professional development systems, such as 
those used by the National Board for Professional Teaching Standards 
(NBPTS). The teacher educators at AS HP sought to design the WBA PD 
activities and the associated badge system to support a sustained program 
of incremental changes to instructional practice so as to help teachers 
incorporate a focus on literacy skills into their regular history and social 
studies instruction routines. 

Many questions about badges remain to be investigated in future 
studies, among them whether badge systems can be acceptable to 
professional educators and school administrators as possible solutions to 
some of the longstanding shortcomings in teacher PD. Digital badge 
systems like this one (i.e., that include research-based features associated 
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with effective PD), however, could enable professional learning programs 
to credential inservice teacher development systematically and over time, 
focusing on gradual improvements to teaching practice in a specific 
subject area. The WBA program accommodates teachers' immediate 
practical needs for classroom-ready materials, while promoting long-term 
development toward instructional practices that align to AS HP’s vision of a 
"master history teacher.” 

Whether that vision accords with the priorities of individual teachers 
and school districts depends on contextual factors well beyond the scope of 
this study, but knowing will be critical to determining whether badges 
become acceptable, or even valued, as PD credentials. For teachers, badges 
could generate opportunities to engage in multiple forms of high-quality 
professional development that allow for individual pursuits within a 
broader framework of standards-based professional learning. This study 
contributes knowledge about how a digital badge system integrated into a 
competency-based, content-specific PD program affected teachers’ 
experiences with and perceptions of the activities. 

Research questions 

Digital badges serve different purposes depending on the setting in which 
they are used, as well as who is using them. Hence, an appropriate 
description of this work is, “a study of a digital badge system as a set of tools 
to support teacher professional development.” As noted, our main question was, 
“In what ways do the features of a digital badge system support or impede 
history and social studies teachers’ progress toward mastery of a new set of 
practices in a competency-based professional development program?” Two 
specific questions guided the data collection and analysis: 

1. What role does working toward and achieving WBA badges play in 
the following aspects of social studies teachers’ professional 
development: continuing participation in the project; peer 
perceptions of a teacher’s professional capabilities; school and 
district administrators’ willingness to treat badges as professional 
credentials; and teachers’ interests in developing new skills? 

2. In what ways do badge-related activities (including unit design. 
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peer collaboration, and assessment by master teachers) influence 
the development of an online community of practice? 

Limitations. The WBA badge system was still in development when data 
collection began in fall 2013. Due to differences in the design vision for the 
badge system's functionality, AS HP terminated its agreement with the 
original developer, ThinkDesign, and contracted with a new team, 
SOSBrooklyn, in summer 2013. The situation led to delays in development 
and impeded AS H P’s ability to recruit teachers and produce lessons. 
Further, the EDC research team was unable to conduct formative research 
with teachers beyond the initial phase of research during the original 
development grant, which precluded us from providing the developers 
with teacher-generated feedback on the site to inform design iterations. 

The circumstances contributed to a lower number of study 
participants than the 40 (plus five master teachers we anticipated would 
contribute after completing the PD program) we had originally proposed. 
The limited availability of content (for much of the duration of the study 
only four units were available for download), site usability issues, and the 
challenge of doing the work to achieve badges (which we discuss below) 
prevented any of the study participants from submitting a lesson plan that 
they developed during the study. Thus, we were unable to answer our third 
question, which asked about the impact of WBA P D participation on 
teachers’ ability to develop and teach lessons that integrated Common 
Core-aligned literacy skills into their routines. 

Nor does our sample include a broad range of teachers with multiple 
badges from whom we would learned more about how school- and district- 
specific factors influenced the usefulness of the badge system as a 
professional development tool. Nevertheless, data from the surveys and 
interviews with teachers allow us to provide rich descriptions of how 
teachers perceived the usefulness of a digital badge system in the service of 
their professional development goals. Further, the data enable us — in a 
limited fashion — to discuss the implications of using a digital badge 
system such as this one to support program features that research has 
found to be effective in improving teaching practice. 
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Overview of the report 

In the remainder of this section we discuss the context in which the study 
was situated and describe the analytical framework we used to analyze the 
findings. In Section II we summarize the development timeline of the 
WBA badge system; outline the badge system and review the PD activities; 
describe the participants and our data collection and analysis methods; 
and present findings from the data analysis. In Section III we discuss the 
findings in the context of existing research on the features of professional 
development programs that improve teacher knowledge and practice, as 
well as the implications for scaling up a competency-based badging 
system. Finally, in Section IV we make recommendations for 
improvements to the current version of the WBA badge system and 
identify factors for investigation in future studies. 

Context: (New) digital badges as (new) credentials for a 
(new) form of online professional development in a 
climate of (new) standards reform 

Writing about the general need for education researchers to use more 
comprehensive methods to study the impact of technologies in educational 
settings, Amiel and Reeves (2008) argued, ‘Technology is much more than 
hardware. It is a process that involves the complex interactions of human, 
social, and cultural factors as well as the technical aspects” (p. 31). 
Accordingly, to understand how the features of a digital badge system 
might contribute to teacher learning, it is necessary first to situate the 
innovation (including its affordances and the stated ends for which they 
were designed) in the social and technical contexts of specific school 
systems (Bennet, McMillan Culp, Honey, Tally, & Spielvogel, 2001). 
Similarly, whether teacher professional development programs of any kind 
are successful at contributing to changes in practice is informed, in part, 
by the specific contextual factors (e.g., district policy priorities, political 
investment of administrators, or availability of human and financial 
resources) of the school system(s) in which they are carried out. 

Based on earlier research about organizational change by Bolman and 
Deal (1997), Wiske and Perkins (2005) identified five dimensions that are 
likely to influence whether innovations contribute to changes in teaching 
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practices: human resources (i.e., individual knowledge, skills, and beliefs); 
structures (i.e., organizational features of a school, such as schedules and 
planning time); cultural-symbolic (i.e., the norms, values, and symbols that 
influence meaning-making within the system); political (i.e., the locus of 
authority and decision making); and technical (i.e., the available resources 
and technologies). Wiske and Perkins further grouped the five dimensions 
into two broad categories specifically for educational settings: craft 
variables, which include human resources and relate to individuals' 
knowledge, practices, and beliefs; and context variables, which include all 
other structural factors and focus on the organization(s). Each of these 
dimensions had some influence on the activities in the present study. 

That the technology innovation central to this research was a “digital 
badge system,” rather than individual “digital badges,” is meaningful. 1 : 
Mozilla's Open Badge Infrastructure (OBI) 5 * — the open-source technical 
standard underlying many, though not all, digital badge systems — is 
innovative in that it allows for credentials to be created and granted by 
nearly any organization with access to the Web and staff with the right 
technical abilities, and then displayed openly and across a range of digital 
environments (e.g., Facebook, Linkedln, or Mozilla's Backpack) by 
recipients. Indeed, “ease of transfer," interoperability, and transparency 
are important features that help define digital badge systems and 
distinguish them from existing credentialing systems. 

Recognizing individual instances of teacher professional development 
activity with a credential such as a completion certificate is common, 
however. All states require that teachers participate in some minimum 
amount of PD hours annually to maintain licensure, though whether a 
provider is recognized and the number of continuing education units a 
teacher receives for participation are both usually at the discretion of 
district and school administrators. Further, the focus and quality of non- 


’ : Rughinis (2013) proposed a very useful definition that encompasses both “badge” and “system”: “A 
‘badge’ is a signaled route through an activity system, with an endpoint symbolically marked as a 
noteworthy achievement through a graphic sign, textual descriptions, and, optionally, rewards and 
entitlements. The route consists of a description of required actions, an assessment of completion, 
and a decision on the identity of the earner. A 'digital badge’ links to online displays of the route and 
the activity system provided by the issuer.” (p. 4). 
http://openbadges.org 


EDC|Center for Children & Technology 
10 


school-based PD activities vary considerably from teacher to teacher (Wei, 
Darling-Hammond, & Adamson, 2010). There would be little point in 
investigating the use of badges in professional development programs if 
they were simply a new, more efficient format for transmitting 
information about “business as usual” PD. 

What is unique to the WBA PD program — and significant to the broader 
enterprise of teacher professional development — is that each badge signifies an 
individual instance of teacher PD activity that is linked to a larger, discipline- 
specific system created to build teacher mastery of content and instructional 
practices. The WBA badge system enables an incremental, cohesive 
approach to inservice professional development. Some badges are 
obtainable after a single activity, while others are “prerequisites” that 
constitute a more advanced badge when taken collectively. But each badge 
represents an element of a comprehensive conceptualization of "mastery,” 
as defined by ASHP. 

As with any credential or award, badges might hold personal 
significance for an individual recipient, the importance of which as 
motivators we do not discount. More germane to the present discussion, 
however, is that WBA badges are designed to function as indicators of 
achievement for an external audience of professional peers. That is, in 
principle, the activities and accomplishments symbolized by the badges 
should be recognizable as components of a comprehensive professional 
learning system for history and social studies educators. The learning 
objectives for content knowledge and pedagogy are likely to be familiar to a 
broad range of teachers, as well as to schools and districts. 

Consequently, to evaluate its usefulness as a tool to support 
professional development beyond the personal value to a teacher, a badge 
should be appraised primarily in relation to the other badges to which it is 
connected in the system, as well as to the badge system as a whole, rather 
than on its own. Additionally, the usefulness of the entire badge system, as 
a set of activities to support inservice professional learning, should be 
evaluated in relation to at least two other sets of factors: First, the 
priorities and policies of the schools and districts (i.e., "the school system”) 
in which badge holders and seekers work; and second, the broad set of 
competencies — identified formally or informally and locally or 
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nationally — associated with being a history or social studies teacher. 

The badge system is unique among social studies and history teacher 
professional development programs. It is not the only innovation that 
influenced teachers' experiences during this study, however. In addition to 
the badges, four other features of the WBA program were novel for nearly 
all participants: (i) The idea of badges as “transparent teaching portfolios”; 
(2) The use of competency-based learning objectives; (3) A focus on 
Common Core literacy skills specifically in the service of improving practices 
among history and social studies educators; and (4) The asynchronous format of 
the online PD. We discuss each briefly below, before concluding this 
section with a description of the study’s analytic framework. 

Digital badge systems as teaching portfolios. Badges are electronic 
records that signify the accomplishment of an individually determined 
goal, or the completion of a task that has value to a specific group. Like 
their physical analogs used by institutions such as the military or the Boy 
Scouts of America, digital badges are usually intended to give viewers 
greater insight into the specific nature of the accomplishment that 
warranted earning the badge, unlike more comprehensive credentials such 
as degrees or certificates, which typically reveal little about an earner’s 
specific skills or achievements. As previously noted, “open badges,” a 
particular type of digital badge based on Mozilla’s Open Badge standard, 
enable additional access to a badge holder’s accomplishments by providing 
links to the work products associated with the badge, along with the 
criteria by which the products were judged by the organization that 
granted the badge. 

Badge systems are a novel and relatively unstudied (see Gamrat, 
Zimmerman, Dudek, & Peck, 2014 for a recent exception) method for 
credentialing inservice teacher professional development activities. Like 
the teaching portfolios (i.e., selections of artifacts that provide information 
about teacher practice) to which they are related, however, badges can give 
teachers a structure with which to document their professional 
development over time and enable them to present pedagogical artifacts, 
such as lesson plans or graphic organizers, that demonstrate their mastery 
of content, or that feature their abilities to design and implement lessons, 
or assess student work (Shulman, 1988; Wolf & Dietz, 1998). In school 
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districts or professional learning communities that value the ability to see 
teacher work when hiring or promoting, badges can permit administrators 
a more comprehensive view of a teacher's skill set (Strawhecker, 
Messersmith, & Balcom, 2007; Whitworth, Deering, Hardy, & Jones, 2011). 

Beyond these “showcase” aspects, construction of a teaching portfolio 
can serve an educative function in professional development by providing 
opportunities to document and reflect on practice, especially as it changes 
over time (Borko, Michalec, Siddle, & Timmons, 1997 ; Zeichner & Wray, 
2001). Researchers have found improvements in teachers' assessment 
practices following completion of the teaching portfolios (which include 
videos of classroom instruction, samples of student work, and extensive 
analyses of teaching practice) they are required to submit as part of 
National Board Certification (Lustick & Sykes, 2006). Large-scale teacher 
evaluation systems, such as the Danielson Framework for Teaching 
currently used by the New York City Department of Education, for 
example, allow teachers to use portfolios as part of their overall evaluation. 

There typically is not a specific call to include artifacts from inservice 
professional development activities, however, and portfolios are generally 
associated with preservice teacher education. While evidence suggests that 
the transparency associated with portfolios and the ability to reflect on 
practice as part of creating them contribute to positive outcomes for 
teachers and administrators, using them does not appear to be a 
widespread practice for K-12 educators. 

Competency-based professional development. Educators and 
researchers have long documented the poor quality of many inservice staff 
development activities, which frequently place teachers in the role of 
passive participants (Little, 1994 ). Ball and Cohen ( 1999 ) observed that most 
teacher PD is “intellectually superficial, disconnected from deep issues of 
curriculum and learning, fragmented, and noncumulative... Since 
professional development is rarely seen as a continuing enterprise for 
teachers, it is only occasionally truly developmental” (p. 4). There is 
empirical evidence for features of professional development programs that 
are associated with improvement in teachers’ knowledge and practices, 
however. As we discuss in the Analytic Framework section below, examples 
of effective characteristics from traditional inservice P D models include a 
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focus on teachers' specific subject areas and how students learn that 
content, as well as the use of activities that are sustained and connected to 
teacher practice over the school year (Desimone, 2011; Desimone, Porter, 
Garet, Yoon, & Birman, 2002). 

Inservice PD activities are often unrelated to the complex interactions 
that occur continually in classrooms among teachers, students, and 
content (Little, 2006). In contrast, standards-based professional 
development — exemplified by the previously discussed certification 
process undertaken by teachers who apply for National Board 
Certification — orients professional development objectives toward 
standards that benchmark demonstrable, high-quality teaching practices 
(Darling-Hammond, 1999 ; Ingvarson, 1998; Ingvarson & Hattie, 2008; 
Lustick & Sykes, 2006; NBPTS, 2014). Though they are not synonymous, 
standards-based PD models can incorporate competency-based targets 
because they are tied to widely recognized competencies. 

Time (in terms of advancement based on demonstrated mastery) and 
regular rounds of differentiated, formative feedback in response to a 
learner's specific needs are two features of competency-based education 
programs. They are not typical of inservice teacher PD, however. Because 
“seat time” is replaced by “demonstrated competency," students and 
educators can devote significant amounts of time and effort to a learning 
program. Although PD providers and subject matter associations (e.g., the 
National Writing Project) can offer extended programs in summer 
institutes and yearlong seminar series, they are not the norm and they 
typically do not require the regular submission of materials by teachers for 
feedback from peers and teacher educators. Such a level of commitment 
for inservice teacher education is a significant change for many educators. 

The Common Core Standards. Unlike the student assessment and 
teaching standards established by other professional associations such as 
the National Council for Teachers of Mathematics (NCTM), there 
currently are no equivalent standards in the fields of history and social 
studies education. Their absence is due, in part, to past controversies 
related to establishing U.S. history content objectives at the national level 


http://www.nctm. org/standards/default.aspx?id=58 
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(Nash, Crabtree, & Dunn, 2000). The Common Core Standards and the 
National Council for the Social Studies' College, Career, and Civic Life (C3) 
Framework for Social Studies State Standards 1 1 1 skirt this controversy by 
making little or no reference to content objectives. Rather, they focus on 
disciplinary concepts and practices such as “arguing from evidence," or 
“identifying an author’s point of view" in order to understand her or his 
motive for creating a document. 

Implementation of the Common Core and the attendant state-level 
assessments, which will affect more educators than the C3 framework 
given their broader focus on math and ELA, is left to each state. States and 
districts are moving at their own pace with respect to aligning curricula to 
the standards and tests. Further, because the focus of the literacy practices 
articulated in the Common Core is in ELA (though there are specific 
reading and writing standards for history, as well), not all history and 
social studies educators will approach them with the same urgency. 

Whereas some states began full implementation in the 2013-2014 
school year, others are implementing in the current 2014-2015 school year. 
Differences in uptake, along with ongoing political tensions associated 
with the Common Core, will affect teachers' and districts’ willingness to 
commit significant amounts of time to ongoing professional development. 
That many teachers throughout the country have already participated in 
some form of P D related to the new standards (though frequently not tied 
specifically to a teacher’s subject area) will also influence their motivation 
to persist in a time- and labor-intensive professional development 
programs that ostensibly addresses similar topics. 

Online professional development. As we noted, there is comparatively 
little research on the effectiveness of specific features of online teacher 
professional development programs (Dede, Ketelhut, Whitehouse, Breit, & 
McCloskey, 2009). In a randomized study, Fishman, et al. (2013) found that 
online PD for a new science curriculum could produce the same impact on 
teacher practices and student learning outcomes as face-to-face PD. We 
noted that Moon, Passmore, Reiser, and Michaels (2013) responded to 
those findings with a challenge to ask more research questions about the 


1 1 1 http://www.s0cialsmdies.org/c3 
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affordances of online PD that might contribute to improvement among 
teacher practice and student learning outcomes. 

The difficulty of bringing teacher professional development to scale — 
that is, reaching large numbers of educators across wide geographical 
areas — is frequently cited as a reason to develop online PD platforms and 
online communities of practice, where teachers can engage with peers 
outside the school building. Other scholars have questioned the wisdom of 
investing in online communities that are isolated from the learning 
communities (when they exist) in which teachers work, however; namely, 
their schools and districts. Schlager and Fusco (2003) argued that, rather 
than focus solely on building virtual communities, online technology 
should first be used to strengthen teachers’ local communities. 

Given existing research that shows that professional development is 
likely to be more effective when it is aligned to local initiatives and 
priorities, that argument is compelling. The method of credentialing 
(badges) and the format of the online professional development 
(competency-based) are likely to influence whether teachers become 
interested in the program and whether they persist, as well as whether 
administrators are willing to support the work. 

Analytic framework 

Before presenting our findings in the next section, we describe the 
framework we used to analyze teachers’ perceptions (via survey responses 
and interview comments) of whether and how the WBA digital badges 
supported their professional development goals. We used five research- 
based “core features” of effective learning in professional development. 
Desimone, Porter, Garet, Yoon, and Birman (2002) identified the features 
in a national study about PD programs that contributed to changes in 
practice for math and science teachers; each is described in Table 1 below. 
As summarized by Little (2006), the research about these features suggests, 
“that teaching to high academic standards requires subject knowledge/or 
teaching. This pedagogical content knowledge is most effectively developed 
through professional development that combines a number of key 
features. Effective professional development is content-focused, active, 
collective, coherent, and sustained” (p. 8, emphasis in original). 
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The framework enabled us to analyze features of the WBA system — as 
represented by specific materials, activities, and technology affordances of 
the program — and their impact on teachers' PD experiences during this 
study. Further, we used it to learn more about the connections, where they 
existed, among the five features and the digital badges for the nine 
teachers whom we interviewed. The badge system is both a feature (a sixth 
feature) and the infrastructure that structures the iterative nature of the 
competency-based professional development program. 

While the lesson materials are freely available, teachers can only 
achieve PD badges by submitting materials, communicating with peers, 
reflecting on their work, and responding to feedback from the teacher 
educators at ASHP. Hence, the badges provide an organizing structure for 
progression through the competency-based system. Table 2 below 
summarizes the specific features of the PD activities, materials, and 
badges that align to each of the core features. 

Table 1: Evidence-based "core features” of PD that contribute to improvements in 
teacher practice (adapted from Desimone, 2011; Desimone, Porter, Garet, Yoon, & 
Birman, 2002; Little, 2006) 


PD feature 

Description 

1. Content focus 

A sustained focus on a teacher’s subject area, connected to standards, 
curriculum, instruction, assessment, and knowledge of how students 
learn in that content area 

2. Active 
learning 

Teachers should be actively involved in the PD activities, engaged in 
activities such as looking at student work, receiving feedback on 
teaching, giving feedback to peers, or participating in lesson studies 

3. Duration 

PD activities should be sustained over time and focused on content, 
curriculum, and student activities 

4. Collective 
participation 

Teachers from the same grade level, subject area, or school should 

engage in PD activities together 


PD feature 
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Description 

5. Coherence 

PD activities should be consistent with other professional 
development, existing knowledge and beliefs, and with school, district, 
and state reforms and policies 

For our purposes, we distinguish between two types of coherence: 
"internal coherence,” which refers to (a) whether the activities for 
each badge are logically related to other badge activities in the system, 
and (b) whether the progression logic of the badge system comports 
with a teacher's understanding of developing expertise in history and 
social studies education; and "external coherence,” or the extent to 
which the activities that the badge system represents are consistent 
with and recognized by the institutional norms and logics and 
priorities of schools, districts, and states 


Table 2: "Core features" of the WBA program 


PD feature 

WBA PD and badges 

1. Content focus 

Inquiry units that focus on U.S. history content and disciplinary 
literacy skills; activities that target improving teaching practices for 
history and social studies teachers; badges reflect a focus on 
instructional design and pedagogical content knowledge specific to 
history education and were generally recognizable to participants as 
important aspects of mastery 

2. Active 
learning 

Teachers teach and modify lessons; annotate student work; reflect on 
implementation; respond to feedback; and plan for future lesson 
implementations. Badges reflect achievements related to these 
activities 

3. Duration 

Inquiry units cover various periods of American history (though 
limited to four for most of the study). Badges reflect iterative nature of 
lesson design and changes in instructional routines 

4. Collective 
participation 

Online teacher forum allows teachers to ask and respond to questions 

5. Coherence 

PD activities address the need to improve teachers' practices related to 
literacy instruction, via the Common Core (“external coherence''); as 
described in the Content focus feature above, teachers recognize WBA 
activities as related to the work of history and social studies education 
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II. METHODS AND FINDINGS 


About the American Social History Project 

ASHP's traditional PD model. The format of ashp’s longstanding 
professional development programs helps explain the evolution of the 
digital badge system. Many of the organization's programs are designed to 
promote a sustained focus on content and practice over time. AS HP 
convenes teachers regularly throughout the school year to address issues of 
classroom practice together that are specific to history and social studies 
educators. In face-to-face meetings, teachers work in teams (often in grade 
bands, though not exclusively) to review activities and primary and 
secondary sources and make adjustments to language and supports given 
what they know of their own students' needs. 

The history educators, master teachers, and historians at AS HP have 
worked with I<-12 and college instructors in face-to-face professional 
development seminars in the New York City area since 1989. AS HP leads 
in-person professional development workshops for grades 5-12 U.S. 
history and social studies teachers, each of which is between 90 minutes 
and six hours long. They also run weeklong summer institutes and 
monthly workshop series throughout the school year. 

ASHP’s workshop and summer institute activities emphasize peer-to- 
peer collaboration on the use of primary documents, social history, 
inquiry-based learning, and literacy building strategies in text and visual 
media. All of their workshop resources are freely available to classroom 
teachers through HERB,*** ASHP’s online archive of primary and 
secondary sources and teaching materials. Finally, as digital media 
producers, AS HP integrates the thoughtful use of technologies such as 
digital games into its professional development workshops. 

The format ofASHP’s professional development activities integrate 
several elements that Desimone (2011) and Desimone, et al. (2002) 
identified as central to programs that contribute to improvements in 
teachers' knowledge and practices, including basing activities in a 


1 1 1 http://herb.ashp.cuny.edu 
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teacher's subject area; a participatory role for teachers during PD activities; 
and a sustained focus on changing teaching practice over time. Because of 
their attentiveness to these features, as well as their emphasis on the use of 
high-quality, classroom-ready, subject-specific teaching materials and 
their longstanding relationship with the New York City Department of 
Education, ASHP has developed a “corps” of dozens of elementary, middle, 
and high school history teachers who return for ongoing professional 
development annually. 

The WBA professional development model. In 2011, ASHP began 
aligning its teaching materials and professional development activities to 
the newly adopted Common Core State Standards for English Language 
Arts & Literacy in History/Social Studies, Science, and Technical Subjects 
in New York State. Anticipating the changing Citywide Instructional 
Expectations — which set the NYC Department of Education's annual 
instructional priorities — the teacher educators worked with middle and 
high school history and social studies teachers during their workshops to 
create “instructional bundles” of primary and secondary sources, activities, 
and graphic organizers to address the new reading and writing standards. 

One goal of these “mini curriculum design workshops” was to produce 
materials that history educators could use to satisfy the DOE's requirement 
that students experience Common Core-aligned instruction across subject 
areas beginning in the 2012-2013 academic year. Another goal was to help 
teachers maintain a focus on “disciplinary literacy” (Shanahan & 

Shanahan, 2012), or “an emphasis on the knowledge and abilities possessed 
by those who create, communicate, and use knowledge within the 
disciplines” (p. 8) as they integrated a focus on reading and writing 
instruction into their existing content-area teaching practices. 

The purpose for creating the WBA badge system was twofold: (1) To 
enable history and social studies teachers beyond the NYC-area to 
participate in PD focused on improving instructional practices specific to 
those educators, using high quality materials; and, (2) To develop an online 
professional community for history teachers across the country to engage 
one another around instructional materials. 

Similar to the face-to-face workshops, the activity focus of the online 
PD activities was in the teaching materials. The inquiry units (collections of 
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5-6 lessons, each of which has a content and a literacy skill focus) created 
by AS HP are examples of sequenced instructional activities that help 
students build and practice, over time. Common Core-aligned literacy 
skills to engage in historical thinking. To progress through the PD system, 
teachers follow a general routine: download existing lessons from the WBA 
web site and modify them based on students’ needs; teach the lesson and 
share modifications and annotated student work with AS H P; reflect on 
lesson implementation in writing; and respond to feedback from AS H P by 
making additional adjustments to the lesson plan as needed and planning 
for future implementations of the lesson plan. 

Because the PD objectives are competency-based, there are no time 
constraints placed on teachers for completing these activities. As the 
professional development model is conceived, reflecting on teaching and 
using feedback from AS HP and peers to make changes to practice and 
instructional materials are the essential activities that contribute to 
mastery. The badges are tied to specific skills, but they are also tied to the 
ideas of iterative design and continuous pedagogical development. Hence, 
the badge system comprises two main types of badges: those that 
recognize “one-time” achievements and those that require several 
iterations of lesson modification and teaching. 

About the Who Built America? digital badge system 

As pictured in Figure l below, the WBA badge system consists of 12 badges: 
11 organized into three areas of mastery — Community, Specialist, and 
Builder — and one Master History Teacher badge. Together, the three 
categories represent AS HP’s conception of excellent history teaching. 
Educators receive Community badges by contributing to the health of the 
online professional learning community, through actions such as sharing 
useful teaching materials, giving peers constructive feedback, or 
commenting and responding to others in the teacher forum. To earn three 
of the four Specialist badges, teachers must create and share a mini-lesson 
that integrates technology meaningfully or helps students focus on a 
specific Common Core reading or writing task (earning the fourth 
Specialist badges. History Geek, requires teachers to score 100% correct on 
four of the quizzes at the beginning of each inquiry unit). Teachers receive 
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Community badges based on positive feedback (e.g., pressing a “thumbs 
up!” button) from other teachers and teacher educators, while only teacher 
educators at AS HP can grant Specialist badges. 


BUILDER 


ALL BADGES REQUIRED 


Preparation, practice, reflection, and revision 
are the key to developing effective units 
and making them better. Teachers 
progress through those steps 
using the Who Built 
America inquiry units 
provided then move on 
to creating their own, 
all with support and 
feedback from 
expert mentors. 



COMMUNITY 


TWO BADGES REQUIRED 


Skilled teachers actively engage 
in their own professional growth: 
they learn from peers, share 
resources, and talk through 
instructional problems 
with colleagues. 


SPECIALIST 


TWO BADGES REQUIRED 


Teachers go deep and 
develop their own areas of 
expertise in history teaching 



ff 

/ MASTER \ 

HISTORY 

WEACHER^ 



Figure I: The WBA badge system 


Unlike the previous two badge categories. Builder badges are 
incremental and hierarchical. That is, to receive an Apprentice Builder 
badge a teacher must complete four steps, as illustrated in Figure 2 below: 
(l) “Know your stuff”: Download an inquiry unit, study the materials, and 
take a quiz; (2) “Get ready to teach” — Share modifications to the unit 
materials, as well as instructional goals, with the teacher educators and 
teach at least two lessons from the unit (including a lesson with a student 
writing product); (3) “How did it go?” — Reflect on lesson implementation 
and share two pieces of annotated student work with the teacher 
educators; and (4) “For next time” — Share modified unit materials and 
“lessons learned” based on implementation reflections with the teacher 
educators. At each stage, teachers submit materials via a series of online 
forms (see Appendix C for an example). Only the teacher educators (and, in 
principle, master teachers, though there were none during the study) can 
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approve teachers' progress toward a Builder badge, as well as award the 
three Builder badges. 



Figure 2: Builder badge progression 

Finally, to receive a Master History Teacher badge, a teacher needs to 
earn at least two Specialist badges, two Community badges, and the 
Apprentice, Journeyman, and Master Builder badges (for each of those, 
they must have completed the four steps above). Teachers must also 
submit a new inquiry unit, of their own design, and have it approved by 
teacher educators or master teachers. ASHP originally expected that it 
would take teachers an average of 40 hours, above the time spent teaching 
materials, to achieve a Master History Teacher badge (see Appendix B for a 
sample teacher profile page). 

Methods 

Recruiting procedures. Following the site's soft launch in November 2013, 
we recruited U.S. history and social studies teachers using the following 
methods: (1) Posting the study announcement on teacher discussion 
forums on the web sites of Edmodo, the National Council for the Social 
Studies (NCSS), the American Historical Association (AHA), the American 
Educational Research Association (AERA), and Edutopia; (2) Contacting 
history and social studies teachers with whom we had communicated or 
collaborated on previous studies or development projects; (3) Speaking 
with history and social studies teachers, as well as school and district 
administrators, at the 2013 NCSS Conference in St. Louis, MO; (4) 
Distributing recruiting materials through the Learn NC online 
professional development site based at the University of North Carolina; 
and, (5) Generating an automatic message to all registered users of the 
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WBA site, inviting them to participate in the study. After a brief screening 
process (using an online survey) to determine that teachers would be able 
to complete the participation requirements in one academic year, we 
invited all teachers to participate. Teachers who completed the research 
tasks (submitting a baseline lesson plan; completing a pre-participation 
survey; completing a survey after receiving one Community badge, one 
Specialist badge, and one Builder badge; and participating in a follow-up 
telephone interview) received up to $250.00 in gift cards. 

All site users. Between November 2013 and the study completion in 
October 2014, 356 educators from 41 states and the District of Columbia 
registered for the WBA site. Eighty-one percent (n= 290 ) of registrants 
identified as classroom teachers and the remaining 19% (n=66) included 
small numbers of curriculum and technology specialists and literacy 
coaches, as well as one principal, one assistant principal, and two student 
teachers. Nearly all registrants (n= 337 ) were history, social studies, civics, 
economics, or humanities teachers. Forty-four percent of registered users 
(n=i 58 ) were from New York State (primarily the New York City public 
schools) and the remaining 56% taught throughout the other 39 states and 
DC, with proportionally larger numbers in California (n=24, 7%), Arizona 
(n=i 3 , 3 - 7 %), Massachusetts (n=i 3 , 3 - 7 %), Illinois, (n=ll, 3%), and North 
Carolina (n=n, 3 %). The majority of registered teachers were veterans: 57% 
(n=203) taught in their primary subject area for at least eight years and 36% 
(n=i28) taught for at least 13 years. In contrast, 21% (n=74) taught for less 
than three years. Though it is an imprecise indicator of activity because it 
is impossible to know whether and how long visitors were active, the 
average number of visits for all users was four visits. 

Study participants. Twenty-nine teachers from 13 states participated in 
the study: 11 were from New York State; five from Arizona; two each from 
Kentucky and Iowa; and one each from California, Illinois, Indiana, 
Minnesota, Missouri, New Hampshire, Utah, Virginia, and Washington 
State. All study participants identified themselves as history, social studies, 
civics, or economics classroom teachers, though one was also a grade 
leader and another a department chair. Similar to the larger number of site 
registrants, the majority of study participants were veteran teachers: 72% 
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(n=2i) taught for more than eight years and 45% (n=l3) taught for more 
than 13 years; only 14% (n=4) taught less than three years. Participants 
taught at the elementary, middle, and high school levels. The average 
number of visits for all study participants was eight. The average number 
of visits for badge earners was 25. Finally, the majority of the study 
participants had at least a Masters degree (n=26, 90%) and half of those 
(n=i3) had concentrated in history education. 

We invited all study participants to participate in individual follow-up 
interviews at the end of the study and interviewed nine (the rest declined 
or did not respond to our invitation): five teachers who did not earn any 
badges and four who earned either a Builder or Specialist badge. 

Data collection. We collected data between November 2013 and October 
2014. Data included: (1) Baseline “literacy-focused” lesson plans that pre- 
dated the study; (2) System-generated logs for all teachers who earned at 
least one badge beyond the WBA Member (users receive these automatically 
after registering); (3) Pre-participation surveys; (4) Surveys following 
badge awarding (6 surveys max.); (5) Communication logs from the teacher 
forum; ( 6 ) Communications with AS H P teacher educators via the 
submission forms; and, (7) Follow-up phone interviews with nine teachers. 
Data analysis. We used teachers' pre-participation surveys to describe 
their previous experiences with “generic” PD and Common Core-related 
PD; teaching Common Core-aligned lessons; and previous experience with 
badges/digital badges. The system-generated logs enabled us to quantify 
the types of badges teachers earned. To analyze teachers’ comments from 
the badge surveys and follow-up interviews, we conducted a thematic 
analysis using five “prior-research-driven” codes (Boyatzis, 1998) based on 
the aforementioned core features identified by Desimone, Porter, Garet, 
Yoon, and Birman (2002). We also created a small set of “inductive” codes 
(Boyatzis, 1998) as we reviewed the data and learned more about teachers’ 
perceptions about the badges. 

Findings 

The findings are organized into four sections. First, we summarize 
background information about the study participants based on their pre- 
participation surveys, including information about their previous PD 
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experiences, the Common Core (teaching and PD), and badges. Second, we 
summarize the badging activities for all site users and study participants. 
Third, we analyze findings related to the two research questions using four 
of the “core features” from the analytical framework: content focus; active 
learning; duration; and collective participation. Finally, we analyze 
teachers’ comments that related specifically to the feature of coherence in 
professional development activities. 

/. Participating teachers’ previous experiences with PD, Common Core, 
and badges 

Nearly all of the study participants (n=26) had previous professional 
development of some form related to the Common Core; only 3 (10%) had 
no previous PD on the Common Core. Half (n=l5) had participated in at 
least eight hours of professional development in the past 2-3 years and 28% 
(n=8) participated in 36 hours or more of Common Core PD in the past 2-3 
years; all of those were from New York State or Arizona. Staff from the 
school or district typically led the trainings (n=i5), though external 
professional development providers (n=7) and trainers from state or city 
education departments (n=5) also led sessions. 

Among the most useful aspects of any training they received before 
participating in the study, teachers reported the following: 

• The most useful aspect was strategies on how to use close reading strategies 
in the Social Studies classroom; 

• Learning how to apply the CCSS in the Social Studies classroom. As a 
non-ELA teacher I was struggling with how to still focus on student 
understanding of the content and not just on student's ability to write; 

• Getting to see how the Common Core applies to materials and activities so 
I know what is expected in the classroom; and 

• The dialogue that was created when we met to discuss the language 
associated with the common core. 

Among the least useful aspects, teachers included the following: 

• [We haven’t converted] over to CC totally as of yet, so having to double dip 
at this point makes it difficult; 

• Professional developments that center around content specific strategies 
and can’t be used in multiple classroom scenarios; and 
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• When the trainer overviews them all and says to do the work yourself from 
multiple sources. 

The teachers had mixed feelings and understandings about the 
challenges associated with integrating the Common Core into their 
existing practices, as well as what they thought the relationship between 
the Common Core Standards and history and social studies education was. 
On the former, eight teachers (28%) felt that “My students don't have the 
basic literacy skills” to engage meaningfully in the level of work the 
Common Core seemed to expect. Six teachers reported a “Lack of support 
from my administration” as the primary challenge; six indicated, “Not 
enough resources for me to use”; and six noted, “There are too many other 
demands on my time.” Several teachers added clarifying comments to 
those statements: 

• My lower level students find the work too challenging. 

• At times I feel that content takes a backseat to literacy. It takes a long time 
to find materials and grading lengthy writings with complicated rubrics is 
time-consuming. 

• The curricular demands are quite large based on the amount of content 
and skills that compose the curriculum. 

Of the 16 teachers who chose to respond to a survey prompt that asked 
them to, “Please describe your understanding of how the CCSS relate to 
history and social studies instruction,” only one reflected a negative 
perception. That teacher commented, “Seems that history will be on a back 
burner and we will really be teaching reading but basing things on history 
content.” Other teachers had a more favorable response to the Standards 
and generally saw them as a valuable addition to history and social studies 
education. Three example statements that reflect that outlook are: 

• The Common Core identifies a standard of writing, reading, presentation 
and interpretation skills, which absolutely goes hand in hand with the 
study of history . History is a wonderful, authentic place for students to use 
and practice these skills. 

• When I think about CCSS and social studies I think about Sam Wineburg 
and his book, Reading Like a Historian. We are tasking students to find 
evidence to support the way that they think. In addition to finding 
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evidence, students should he able to view the source and make a 
determination as to what the perspective of the source could be. The CCSS 
is looking for students to drill deeper into the material, so that a stronger 
basis for opinion, argument, point of view can emerge. 

• No longer are students going to be memorizing discrete historical facts. 

The "endgame" is for students to be able to read, understand, analyze, 
synthesize, and write argumentatively using evidence regarding historical 
sources, both primary and secondary. 

Broadly, the teachers who participated in the study had positive 
associations to the Common Core Standards and their role in adding to 
history and social studies education. Most had previous PD related to the 
Standards and found it useful, though approximately one third felt that 
their students were not prepared to engage in the complex literacy 
practices targeted by them and others felt they did not have time, 
administrative support, or resources to focus on them. In general, the 
teachers did not see the Common Core as replacing history content, but 
rather providing students with means to engage in the complexities of 
historical reasoning and supporting explanations and arguments with 
evidence. 

Finally, the majority of teachers (n=i9, 66%) had no previous 
experience with badges of any kind before the study. Of the to who did, 
most suggested that badges were significant because they symbolized a 
personal achievement, but that they did not necessarily hold value for 
others. For example, one teacher commented, “Personally, the badges have 
not had any real significance for me; however, once earned, it is nice to be 
able to speak of the achievement.” Another noted that badges “help check 
things off a to-do list.” Another wrote that her badges were, “Significant 
because they represent commitment to and completion of a task.” Only two 
teachers described badges as a means to communicate achievements with 
others: one who liked that others in her Weight Watchers program could 
monitor her weight loss with her and another who found a badge useful as 
a way for creditors to track his credit rating. 

II. Summary of badging activities for all participants 

Table 3 below displays the number of badges awarded by AS HP for each 
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type. A very small number of users achieved a badge during the study. All 
educators received a WBA Member badge once they registered for the site, 
so we do not count those toward badges earned. Only seven percent (n=2) 
and one-and-a-half percent (n=5) of study participants and non-study 
participants, respectively, earned an Apprentice Builder badge. 

The total number of badges acquired by an individual means 
comparatively little, as there are qualitative differences between the efforts 
associated with achieving different badge types. For example, the two 
study participants who achieved Apprentice Builder badges committed 
significant amounts of time beyond classroom teaching time to complete 
the work for the badge: 8 hours and 25 hours, respectively. In contrast, 
another teacher took a total of two hours to earn two Specialist badges: a 
Reading Specialist and a Writing Specialist badge. Like quantity, time 
spent on a specific badge is also an imperfect indicator of its value as a 
professional development activity, particularly in a competency-based 
system that eschews a “time on task” orientation for mastery. Nonetheless, 
some badges do require more effort than others. 

The percentage of educators who earned a badge of any type is low. In 
the following sections, we use the analytical framework to explain why this 
might be so, discussing positive and negative experiences teachers had 
with each feature. In the final analysis section on “coherence,” we return to 
participants' comments about specific badges in order to analyze the 
“internal and external” coherences between the features of badge activities 
and teachers’ goals and districts' policies. 

Table 3: Number of recipients by badge title 


Badge title 

Study participants/ 
Average n of hours 
to achieve badge 
(n=29) 

Non-study 

participants (n=327) 

Apprentice Builder* 

2 ( 7 %): 16 hours 

5 (1.5%) 

Journeyman Builder 

0 

0 

Master Builder 

0 

0 

Master History Teacher 

0 

0 

History Geek 

5 (17%) : N/A 

5 (l.S%) 

I Love Sharing 

4 (14%): 1 hour 

3 (.9%) 
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Badge title 

Study participants/ 
Average n of hours 
to achieve badge 
(n=29) 

Non-study 

participants (n=327) 

I Love Community 

3 (10%) : 1 hour 

2 (.6%) 

Common Core Reading Specialist 

1 (3-S%): 1 hour 

1 (.3%) 

Common Core Writing Specialist 

1 (3-5%): 1 hour 

0 

Constructive Critic 

0 

2 (.6%) 

Tech Guru 

0 

1 (.3%) 


* Builder badges consist of four “sub-badges,” or “stars” (See Appendix A and the section 
on the WBA badge system on page 20 for a fuller description.) 

Ilia. Question 1: What role does working toward and achieving WBA 
badges play in the following aspects of social studies teachers’ 
professional development: continuing participation in the project; peer 
perceptions of a teacher’s professional capabilities; school and district 
administrators' willingness to treat badges as professional credentials; 
and teachers’ interests in developing new skills? 

We discuss positive and negative comments from teachers about three of 
the core features that help answer this question: content focus, duration, 
and active learning. In general, the teachers with whom we spoke 
appreciated the content focus but found the level of effort required to 
achieve badges to be too much, even if the effort could be extended over 
time. We address teachers' comments that related to “badges as 
professional credentials” and peer perceptions in a separate section on 
“coherence” below. 

Content focus +. The teaching materials were the most compelling feature 
of the WBA site for all of the teachers whom we interviewed. Nearly all of 
them commented on the quality of the primary and secondary sources (in 
terms of their usefulness in the classroom), as well as the lesson plans in 
the inquiry units. As one teacher put it, “Like all teachers. I'm always 
looking for stuff I can use in the classroom, either tomorrow or next week. 
And this was really good stuff that I could use right away.” Seven of the 
nine teachers said that they were very likely to use lessons from the WBA 
site again next year. 

Five of the nine teachers mentioned that the materials would help 
them address the need to integrate a focus on literacy practices into their 


EDC|Center for Children & Technology 
BO 


teaching. One teacher in Washington State commented, ‘We're trying to 
have our students here use primary source materials more. I liked how on 
the WBA site they had the units built up on different topics, and that it 
included the Common Core standards. It was killing two birds with one 
stone: content and Common Core.” Another teacher, in Kentucky, noted 
that, “I am slowly adopting my lessons to resemble the Common Core. It's 
been adopted in every other subject area, so I’m adopting my lessons to 
meet that. With your materials. I’ll be prepared, so there will be less of a 
transition for me.” And a teacher in New York City said that, 

I definitely liked the fact that there was a lot of Common Core alignment here. 
That’s a big movement in the city and the state. I find that the most useful 
professional development does one of two things for me. It has to be focused 
directly on what I’m teaching at the time, as opposed to something I’m 
teaching in the future or the past. Otherwise, it gives me time during the PD 
to connect to my own materials and to share it with other people so that I 
could get feedback. This PD seems to be doing both those things. 

Four of the teachers made little or no reference to the Common Core when 
we asked them what drew them to the site. A teacher who was most drawn 
to the historical content said, “The quality of the materials is great — it’s 
what drew me to the site. I love to teach with images, being smacked in the 
face with that wonderful image of the women. I’m in my 8th year of 
teaching and I use a lot of primary sources. I knew that you would give me 
access to primary sources lessons.” 

None of the teachers referenced a connection between badges and the 
historical content, though as we noted, five did connect learning about the 
Standards to the badges. The two participants who earned an Apprentice 
Builder badge mentioned reasons that had nothing to do with content 
when we asked why they persisted to earn the badge, though both valued 
the content. One teacher persisted because “I wanted the stipend, but I 
would have done this anyway. I do a lot of PD.” Another said, “That’s just 
me. I have to finish what I started. It’s an internal motivation thing.” 

Content focus -/Duration Two aspects of the WBA program’s in-depth 
focus on historical content were a hindrance for all of the teachers with 
whom we spoke, however: the relative lack of content to cover the entire 
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U.S. history survey and the scope of the existing inquiry units. For much of 
the year there were only four units from which teachers could choose, 
which left several participants in the position of having passed points 
when they could use the content with their students. Being unable to teach 
those units also left them in the position of being unable to pursue Builder 
badges. For example, one teacher commented, 

Here is the difficulty— your timeline is not the best timing for me. The badge 
that had to do with the American Revolution.. .once I visited the site, got 
familiar with it, I was already halfway through the Revolution, so I wasn’t 
going to be able to use that to work on the Builder badge. At that point I 
thought my best approach to this was to go on to a content area that I had 
time for and thafs when I chose the Lowell girls unit. But as we were going to 
that, that’s when we hit inclement weather. If there was a way that I could 
have received materials ahead of time, I would’ve worked on them. But when 
things were available, I had already passed it. 

Another teacher faced a similar situation, saying “I only found out about 
the site after I had already taught the things that were there. I couldn't 
submit any student work because I couldn’t teach it. A lot of those units 
didn’t apply to me because I couldn’t teach them. So, I couldn’t get any 
badges.” 

The scope of the inquiry units was also off-putting for several teachers. 
One teacher in Utah commented that, “There’s too much. These units are 
gigantic. I have two days to teach the Industrial Revolution, but this thing 
could take two weeks. The mill girl thing, I didn’t even know how to start to 
pull that apart to use it in my classroom. I would like to see smaller units, 
smaller chunks. Eventually, I hope that you will continue to have more 
units, but shorter, that will fit into my work.” 

The challenge of the time it took to teach a WBA unit concerned several 
of the teachers. One of the Apprentice Badge earners said, “I didn’t really 
have any grand schemes for this. You just had to start early and then keep 
going with the reflections afterwards. In the War on the Philippines [unit], 
I thought ‘I’d like to do this,’ but I also thought I wouldn’t have time to do 
everything. It just didn’t fit into the time I had in the scope and sequence.” 
Another teacher also felt that the units would take too much time, but that 
he would be in a better position to use them next year, 


EDC|Center for Children & Technology 
32 


After my second time around with this. ..If I could have compacted the content 
or the activities into a smaller time span to use in my classes that would have 
worked out this year. For next year... I would take a week, assuming there are 
no unusual delays...I’m familiar with our state tests, so I know what I can 
compact, that way I could have larger units on other content areas. 

Active learning +. Six of the nine teachers told us that they felt the 
activities they had engaged in during their participation were valuable to 
their professional growth. Further, several felt that the level of activity 
expected of teachers in WBA was unique, and positive, among their 
experiences. Three of those teachers — one of whom earned an Apprentice 
Builder badge — commented specifically on the cycles of reflection and 
feedback that were built into the submission processes to earn badges. The 
badge earner commented that, 

The reflection piece is awesome. You know, we learn from reflecting. It’s a 
really positive thing. It [the PD program] actually makes you use the 
strategies and do the activity that you learned about. In a lot of inservices you 
tell yourself, ‘that’s a great idea,’ but you don’t do it. But in this one, you 
actually had to do the lesson in order to advance, in order to move on. I really 
like that about this. 

Another teacher in California said that, 

I earned a couple of badges, the reading, writing, and tech specialist ones. 
There’s so much for me to learn right now — last year we started implementing 
it [the Common Core]. Right now we’re being inundated with stuff. But this 
project actually gave me feedback. That is so valuable. There’s no other PD 
I’ve been given that gets me the kind of feedback I got here. You know, in other 
training that I got, it didn’t emphasize counterarguments. But that was 
emphasized with badges. When I got feedback from them, I realized that I 
skipped over that, that my kids weren’t getting it — that feedback got us right 
back on track to hit all those standards. 

A third teacher commented, “The supplemental enhancement activities 
that those [Specialist] badges gave, that was what generated the most 
interest for me. I’m not going to say that an old dog can’t learn new tricks, 
because I think I learned some new stuff thinking about those badges.” 
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Lastly, a teacher in New York City said, “If it's specific and accurate 
feedback, then it's extraordinarily helpful. The advantage that this PD has 
over college classes is that the learning stops in college classes when you 
get a grade. But here, you're getting feedback on your work from the 
classroom. It's extremely helpful.” 

Active learning The level of teacher activity was also a drawback for 
some teachers, however. Several teachers spoke at length about how much 
time this project seemed to demand, even among those who felt positively 
about being required to submit material and respond to feedback. As one 
teacher put it, “Look, for this thing to be successful, the main thing that 
needs to be taken into consideration is that teachers are so extraordinarily 
busy. You have to take their time into consideration for it to be 
meaningful. This work is good and important for development, but there's 
so much other stuff to do.” Another said, “But this is really time 
consuming. You have to pre-plan, pass the badge, make sure you have that 
many days in the schedule, and then remember to go back to it to submit 
whatever else you have to submit. You only have 24 hours in the day. It's a 
lot of work, this PD. And I've already got a lot of work." 

One of the teachers who earned an Apprentice Builder badge said that, 
“It took me 25 hours to get that Apprentice badge. I always have to do a lot 
of extra work anyway because I have to translate things back and forth 
between English and AS L. But I wasn't going in to this planning to do a 
unit. It was a lot of unexpected work. I've put that much work into a 
weeklong unit before, but I wasn't expecting to do it with this.” Another 
teacher, who taught for more than 20 years, remarked, “Whatever else this 
badges project does, it has to accelerate something you're already involved 
with. We have so much on our plates and this felt like a lot of busywork. I 
didn't see where this was going for the badge people or for me. Why is this 
a big deal for you? Sometimes it felt a little like a black hole of 
nothingness.” 

The teacher's question about ‘Why is this a big deal for you?” extended 
to several others, as well. While many enjoyed the opportunity to receive 
feedback on their work, they also felt that it was not always timely or clear. 
For example, one teacher who had submitted work for a Specialist badge 
said, “The feedback was, You kind of missed the mark here.' I wrote her 
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back and said, ‘you're right.' But it took like two and half weeks to get a 
response. It was too late. I couldn’t submit it because I'd already taught it. I 
was very frustrated.” Another teacher who had submitted material for a 
Specialist badge commented, “I couldn't understand the feedback. I didn’t 
see how I was off the mark and I couldn’t get why this wasn't good enough 
for a badge. A rubric would’ve helped.” 

Illb. Question 2: In what ways do badge-related activities (including 
unit design, peer collaboration, and assessment by master teachers) 
influence the development of an online community of practice? 

All of the teachers with whom we spoke felt that the online teacher forum, 
in its current form, was not useful as a place to support professional 
growth. Most felt that there was not enough useful conversation for them 
to attend to, or that the teachers who had posted there had done so strictly 
to achieve a badge. Two of the four teachers who earned Community 
badges indicated that the badges were “much less valuable” than other 
badges they had earned because of the minimal effort they had required. 

Collective participation All nine of the teachers whom we interviewed 
responded relatively unfavorably to our questions about the value of the 
teacher forum and the Community badges during their WBA PD 
experience. Several teachers disliked these features because they valued 
what they represented in their day-to-day professional lives, while others 
felt that there was no meaningful connection between the Community 
features and other parts of the WBA PD program. Some teachers also 
recognized that the site was new and that it would take time for a teacher 
community to develop, however. As one teacher remarked, “Working with 
a team would have been more motivating than doing it by myself. But 
that's not their fault. They have community forums, but there's not much 
posting going on.” 

Another teacher said, “I'm not interested in the message boards. I have 
enough online sites that I go on and this doesn’t add anything.” Another 
commented, “The Community forums? I looked at them but it did not 
seem like a lot of action was going on, so I didn’t participate. It looked like 
people were posting just to earn badges. I might post or read if it was more 
active.” Similarly, another teacher remarked that, “It felt like people who 
were posting were only posting the bare minimum to get the badge.” 
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One of the teachers who had earned an Apprentice Builder badge also 
felt that working with colleagues would have made some of the 
requirements for earning badges less burdensome. He said, 

I don’t know how to improve the inquiry units — they’re great. But I did make 
one comment in the Internet surveys about the Common Core Writing and 
Reading badges, where you have to make up your own units. I thought it’d be 
good to do professionally. It’s a ton of work, which is good, but if you could 
team up with other people, you could propose a topic and get a group of people 
to do it together with you. 

The same teacher reflected on what he thought was lost in not having 
colleagues with whom he could work on materials. He remarked, 

As opposed to sitting down together at the end of the inservice and looking at 
the teaching strategies together, in this one [WBA] I think that it can come 
across as just printing off activities and then just doing them, which doesn't 
necessarily bring any change to teaching practice. In the more interactive PDs, 
I can think about how to apply this [the strategies] to my own practice. I 
mean, in face-to-face, usually there’s a scaffolded writing assignment. The 
coaches have you and the people at your table do something together. The 
difference there is that it has this sense of, You struggle with it yourself, "as 
opposed to ‘Do it yourself’ There’s an advantage to doing it with other teachers. 

IV. Coherence with other professional development, existing knowledge 
and beliefs, and with school, district, and state reforms and policies 

We distinguish between “internal coherence” and “external coherence” in 
the analysis, but both relate to the degree to which activities in the badge 
system align to perceptions, norms, priorities, and values associated with 
history and social studies teaching, either for an individual teacher or an 
external agency such as a school or district. 

Internal coherence +. Internal coherence is the degree to which PD 
activities in the badge system seemed logically connected to each other, as 
well as the extent to which the progression logic (i.e., how teachers advance 
by achieving badges) reflected teachers' personal understanding of what it 
meant to develop expertise as a history teacher. External factors are likely 
to inform the latter and so the differences between “internal” and 
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“external” coherence are guidelines, rather than rules. The distinctions are 
useful for analysis, however. 

Six of the nine interviewees indicated that the badge system “made 
sense as professional development credentials,” even if they were not 
personally interested in using the badges as such. For example, one teacher 
said, “Yes, they made sense to me, but their importance to me was. ..and 
what I’m doing.. .and what I’m required to do in my school.. .did not really 
jive together.” Another remarked, “The idea of earning a digital badge, it 
seemed kind of odd. But I talked with co-workers who said “Oh yeah, you 
can earn badges for different things you do on the Web, like in games. So, I 
get how you can use them to show that you accomplished something.” 
Comments by the other two teachers suggested that, while several 
activities related to specific badges made sense and appealed to them, they 
did not see how the badges connected meaningfully as a professional 
development sequence. 

An important theme related to internal coherence was “sequence and 
progression." Five of the seven teachers for whom badges seemed viable as 
professional credentials made comments that suggested an appreciation 
for how the badge system “chunked” (though not always well, as comments 
above suggest) PD activities and portrayed professional development as a 
progression. For example, one teacher said, “I like how you had to get a 
certain percentage to get the [History Geek} badge. I had to take a few 
quizzes more than once... I thought that was great. ..To move on to the next 
step, it doesn’t allow you. You have to unlock a certain level, it’s sequenced.” 
Another remarked that, “I could see a gradual progression through them. It 
seemed very doable. I got excited. I thought that I wanted to take the time 
to do it.” Another said that, 

I did like the concept because it felt like, particularly when teachers attend PD, 
they throw the whole buffet at you at one time. But you all [AS HP] present it 
in steps. You give us breakfast, then a snack, then lunch, then dinner. It’s 
helpful by breaking it up, there’s a feeling of success. By the time you’ve come to 
the last batch, you feel like you’ve achieved something. I felt like, ‘Even if you 
didn’t make Eagle Scout, you’re still a Boy Scout.’ I like that concept. 
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Internal coherence Comments from three teachers suggested that the 
WBA badge program lacked internal coherence for them. One, a veteran 
teacher of more than twenty years, commented that, Tm an older 
educator, Fve been around the block.. .Not that I'm afraid of incorporating 
new things, technology, or new ideas. I have no problem with adding, 
changing, or keeping up. ..but there's not enough time in my life. I’m not 
one who's into accolades." Remarks from the other two teachers indicated 
that the level of work required for earning a Builder badge was not in line 
with their own priorities for PD. 

External coherence. External coherence refers to the degree to which the 
PD activities align to school and district priorities for teacher training and 
growth. Three of the four factors identified by Wiske and Perkins (2005) as 
determinant of whether innovations are successful at contributing to 
changes in teacher practice applied to this study: structures (i.e., 
organizational features of a school, such as schedules and planning time); 
cultural-symbolic (i.e., the norms, values, and symbols that influence 
meaning-making within the system); and political (i.e., the locus of 
authority and decision making). Technical was not a factor we addressed 
directly. 

Two issues related most to external coherence: (1) whether schools and 
districts granted credit for badges (they did not); and (2) whether peers or 
school or district administrators valued the finer-grained distinctions 
among professional development activities that badges might permit (they 
did not). Both of these issues had implications for whether teachers 
thought they — and others — would be willing to perform the level of work 
required a competency-based system such as this one. 

One teacher commented, “I think another thing that would incline me 
to do more is that if the badges turned into something useful. If you could 
turn them into PD hours, that would make a difference. They’re fun for the 
pat on the back, but it would be better to have in-service pay or 
professional development credit, that would increase my incentive to do 
this.” Another said, “I like the badge system, it could be fun; but I 
remember going on there and it wasn’t that much fun. It was a lot of 
reading and I remember it being a lot of difficult work. It became an 
extension of my job, rather than something I could do for my own 
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professional development. That much work without getting credit isn't 
realistic.” He added, “Even if there were an opportunity for continuing 
education credit, people might choose other things over this because there 
is so much work. You have to put a lot of work into this.” A teacher from 
Utah said, “I told my principal about them. He was like, ‘Okay, great.' I was 
like, “Where am I going to post these? My personal web site? No. My school 
site? No.' He was like, Whatever.' A professional development badge, as a 
term, has no meaning in Utah.” Finally, one teacher addressed the 
challenge of finding value in badges among other competing factors: 

There has he a value in it. If people don’t see a value in it, forget it...This isn’t of 
value to me because no one has talked to my district about badges. I could show 
them my certificate, and they would say, “Great, what did you do? Fine, put it 
in there. ’’But badges aren’t understood. It would have to be understood by those 
people I report to. I would have to go through WBA badges, contact the 
curriculum director, and talk about the value ofbadges...There’s so much 
expected of us: Common Core, report cards, assessments, trying to get your 
Masters, or recertified. Where does badges fall on there in terms of level of 
importance in our lives?’’ 

A second issue related to external coherence is that most teachers do 
not receive personalized recognition for different forms of professional 
development. Because the norm for most states is seat time, most teachers 
in this study saw little value in presenting credentials that might 
distinguish them from other educators, though some are in schools that 
recognize and value different forms of activity. As a teacher in Minnesota 
remarked, 

It’s more about attitudes toward teaching and learning. Some schools have 
great PIC (professional learning community) networks. Some schools have 
great teams of teacher that reflect on their approaches to teaching. They like to 
try out new ideas. My attitude is that I always have something to learn. Big 
school districts probably only look at your hours, they don’t know you 
personally and they’re unlikely to look at the specific types of PD you do. 

A teacher in New York had a similar comment: “I think it depends on 
the school. In some schools, if you went through a particularly useful PD, 
then some principals will require you to turnkey it for other teachers. In 
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my school there's a desire to do that, but it doesn't happen often. You hear 
about people having PD, but it never gets celebrated in any way. It isn't 
seen as this wonderful thing.'' Another teacher expressed the general 
sentiment about most PD succinctly: “For PD, you just get your certificate. 
You register online, do a 5-question evaluation and rate it. You say what 
you got out of it. Then you submit and print certificate. It's just a set thing 
everyone does, there's no recognition. We have to do what everyone does, 
which isn't very rewarding.'' 
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III. DISCUSSION 

In this report we investigated how the features of a digital badge system 
supported or impeded history and social studies teachers’ progress toward 
mastery of a new set of practices in a competency-based professional 
development program. We used a framework that included five “core 
features” of professional development programs identified in previous 
research as being effective for helping teachers learn and change practices. 

We learned that it is too early to investigate research questions about 
whether badge systems might be effective tools for promoting teacher 
professional development. In the case of this specific badge system, 
development issues slowed the production of nearly all features that might 
have the most impact on teacher development and left the researchers with 
a site that had not benefited from formative research. In the broader 
context of I<-12 teacher education, however, there are no external 
authorities, such as city or state departments of education, that are 
currently granting continuing education units in exchange for badges. 
Given the level of commitment expected of teachers by a competency- 
based professional development program, the lack of formal methods with 
which to recognize that work is likely to inhibit the uptake of this type of 
professional development beyond the adventuresome few who are typically 
“early adopters,” or who engage in many hours of PD above and beyond 
their required hours for recertification. 

The analytical framework of five core PD features was useful for 
helping us to identify the affordances of a badge system that teachers 
might value, should professional development programs like WBA become 
acceptable forms of credentialing. Below, we revisit the features briefly 
and discuss their implications for the ongoing use of a competency-based 
PD system among history and social studies educators. 

Content focus, active learning, collective participation, and duration. 
The majority of the teachers whom we interviewed valued the teaching 
materials above all other aspects of the WBA site. Indeed, searches for 
teaching materials were often “the way in” to finding the project and 
becoming aware of the professional development program. A smaller 
number of teachers commented directly on the value of these materials for 
helping them to learn about integrating the Common Core, however, and 
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we are unable to report on how well teachers did so because none 
progressed to a point where they could submit new lesson plans. 

The ability to reflect, plan, and receive feedback was very attractive for 
several of the teachers, as most have few opportunities to do it if they are 
not part of a team of teachers in a school that look at student and teacher 
work together. Further, the chance to have useful, timely feedback on 
teaching materials from qualified teacher educators was exciting for more 
than half of the teachers whom we interviewed. Though we had little 
feedback about teachers' experiences with integrating Common Core- 
aligned skills during this study, it was a place where two teachers felt the 
feedback from AS HP was extremely valuable. Both felt that their practices 
had improved based on that exchange. 

All of the teachers commented on how challenging it was to complete 
all of the requirements for the WBA program, especially as there was not 
enough material on the site for them to use in order to work toward 
obtaining Builder badges over the school year. While several teachers 
appreciated the repeated rounds of feedback from AS HP on their 
materials, the payoff typically did not match the effort, especially when 
there was no formal recognition for the badges forthcoming. 

Our experiences during this study have several implications for future 
projects that might use a badge system to support competency-based 
teacher professional development: 

1. Content focus: Social studies and history teachers value 
classroom-ready material they can use immediately, which the 
WBA site includes. The units are very large, however, and there 
currently are not enough of them for teachers to use 
throughout the school year. If badges are to be tied to teaching 
the content, then the content should be presented to teachers 
in smaller lessons that they can teach more quickly and, 
possibly, more frequently. 

2. Active learning and duration: PD activities that include 
submitting new lesson plans and student work, receiving and 
responding to feedback, and creating new teaching tools are 
intriguing for many teachers. Several teachers in this study saw 
the potential for growth as educators through rounds of these 
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activities and liked that the structure of the badge system 
supported an incremental approach. But they are time 
consuming and require a level of activity that is not the norm 
among current PD programs. To make the rigor and level of 
effort more acceptable, feedback on teacher work should be 
timely and varied (in that it comes from teacher educators and 
peers). Further, as we discuss next, teachers should have 
opportunities to do the work with peers. 

3. Collective participation. Teachers benefit from peers' comments, 
but there are few opportunities to do this on the WBA site. The 
Community badges were the least appealing to all teachers with 
whom we spoke and several found the inability to engage with 
others while doing so much work to be an inhibiting factor. 

Coherence. For most of the teachers whom we interviewed, the WBA 
badge system aligned to their understanding of what social studies and 
history teachers needed from inservice PD in order to keep growing, 
though specific features — as we have discussed — raised questions for 
them as to whether they could persist in the work. The amount of work in 
the program is considerable. But the matter is exacerbated by the fact that 
there are no school systems (or schools) that accept the badges as 
continuing education units. The lack of external validation has important 
implications for whether a competency-based badge system can survive 
and scale when “seat time” is the norm and, in fact, what is generally 
approved by most state systems. 

Earlier we noted the success of standards-based PD programs such as 
the application process for National Board Certification. One way that a 
badge system such as WBA — and others — might become sustainable is if it 
is able to secure for teachers benefits such as extra pay or extra privileges. 
For that to happen, of course, these badge systems will need to 
demonstrate impact on teaching practice and student learning outcomes. 
Hence, once AS HP secures commitments from external agencies to 
support and recognize this type of PD, it should consider other studies to 
investigate the impact of the badge system on teacher and student 
learning outcomes, especially with respect to the new Common Core- 
aligned assessments that states will begin implementing. 
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IV. RECOMMENDATIONS 

1. Create “onboarding" badges to decrease the amount of time it takes 
to earn a “ meaningful " badge 

Seven of the nine teachers whom we interviewed remarked on the 
amount of work it required to achieve a "meaningful” badge, though 
only two of those completed the requirements to achieve an Apprentice 
Builder badge. Those two teachers committed between 6 or 7 hours 
each to completing the work for the badge, beyond the time it took 
them to teach the lesson. 

Other than the WBA Member and History Geek badges, there 
currently are no other "minor” badges that teachers can achieve; all 
other badges in the system require teachers to submit materials for 
feedback or engage in a repeated number of activities, such as posting 
in the forum. One of the nine interviewees (one of the Apprentice 
Builder participants) commented that he liked “racking up badges 
quickly,” though he did not see any advantage in doing so 
professionally. Rather, it appealed to his desire to "get things done, 
once I get going with something.” Given the very low number of badges 
earned among all participants, it is evident that it is very difficult to 
achieve many badges as the system is currently configured. 

None of the teachers with whom we spoke was interested in 
achieving badges for the sake of achieving badges, other than the one 
teacher we discussed above, none of the teachers with whom we spoke 
was interested in achieving badges for the sake of achieving badges. 
Rather, it was the opposite. Two teachers found the History Geek 
badges “insulting” or “a waste of my time.” Those teachers, and others, 
would have preferred opportunities to receive “smaller badges that still 
relate to something I can do with my kids and that I feel good about 
getting, like I put something into it.. .without breaking my back,” as one 
teacher remarked. 

One possibility might be to “chunk” the Builder badges such that 
each of the four required steps becomes “badgeable.” It would require 
altering the activities such that teachers might be able to perform a 
smaller task, yet still one that has pedagogical value. Another possibility 
is to create “minor” versions of the current Specialist badges such that 
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teachers, again, can submit smaller pieces of work but that are still 
deemed to have instructional value. 

2. Identify local AND state- or district-level partners to grant credit for 
badges (or provide teachers with other opportunities, such as 
release time, to participate) 

None of the teachers who participated in this study had confirmation 
from a school or district administrator that they would receive 
continuing education credits for the participation in the WBA project. 
As we noted, five of the nine whom we interviewed did not share 
information about the badges with colleagues, in part because they 
assumed the badges would have little or no cachet. 

This is a larger challenge to confront than our previous point 
because the contextual factors are mostly beyond AS HP's control. We 
recommend partnering with a very small school district — or an 
experimental PD program in a larger district — that could help ASHP 
negotiate the administrative issues related to granting credit. The 
opportunity to receive credit would make the program more attractive 
to many teachers, and it would provide them with additional 
motivation to persist in the challenging work of a competency-based 
program. Testimonials and demonstrations of changes in practice or 
improvements in student outcomes (if they exist) from this type of 
collaboration between ASHP and a school district would provide other 
districts with evidence for the merits of this type of PD. 

Further, local partners could be in a position to promote other types 
of systematic approaches to teacher education (such as supporting 
school-based policies that promote the development of teacher 
learning communities) that accommodate and encourage the 
competency-based PD objectives the WBA badge system targets. 

3. Improve mechanisms to support asynchronous and synchronous 
communications among teachers and teacher educators 

One study participant commented that he found the work of this 
project “to be a little lonely” and several others remarked that they felt 
the communications in the teacher forums were “artificial,” “forced 
because people want the badge,” or “not all that helpful.” Three of the 
nine teachers whom we interviewed suggested that ASHP should 
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dedicate time, either semi-monthly or monthly, to bring teachers 
together on the site at the same time. One teacher suggested “monthly 
lectures” that all WBA teachers could attend and have opportunities to 
ask questions of the teacher educators and each other. Another 
suggested that the teacher educators (or a teacher that had achieved 
Master History Teacher status) should convene small groups of 
teachers that were working on the same unit for an hour every other 
week in order to discuss their progress. 

None of the teachers we interviewed thought the teacher forum 
was helpful. One way to improve the forum might be to restructure the 
discussion boards based on specific units, such that teachers who are 
all working with the same material in different places can ask 
questions about the same materials. At best, these discussion boards 
might be ancillary to lesson studies or the live discussions that we 
mentioned above. 

4. Increase the frequency of communications between the teacher 
educators and registered users 

Several teachers commented that they forgot about the WBA site 
because of the lack of communication from AS HP and only 
remembered when EDC researchers contacted them at points 
throughout the year. Emailing teachers at least monthly with updates 
about new lesson plans or site features would be very helpful. Several 
teachers commented that they would like updates from the teacher 
educators about their specific progress, even if that progress were 
somehow visible on their profile page. Specific instructions about 
“what to do next” would help teachers have clearer ideas about 
upcoming requirements. 

5. Improve the web site layout and print material layout in order to 
make the program more approachable and useful for all users 

Many teachers commented that the web site and print materials for 
lessons felt “cluttered.” One teacher remarked that, “It felt like you were 
trying to get everything under the sun into some of those web pages 
and the lesson plans. It got really overwhelming.” If lessons are 
chunked, as we suggested above, then the print materials will become 
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less overwhelming. Several focus groups with groups of teachers, along 
with think- aloud exercises with individual teachers, would be helpful 
for determining how to structure the site such that teachers can find 
teaching materials easily, locate general information about badges (as 
well as information about their own specific badges), and respond to 
feedback from the teacher educators more quickly. 
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APPENDIX A: BUILDER BADGE CRITERIA 

The Apprentice Builder has started on the path toward becoming a Master 
History Teacher who creates effective Common Core aligned lessons in U.S. 
history. To earn this badge, the teacher: 

• Studied the [unit title] Who Built America inquiry unit and passed a 
multiple choice quiz with a score of [quiz score], demonstrating 
understanding of its historical content and aligned Common Core 
standards. 

• Prepared to teach the inquiry unit by revising lessons and materials to 
meet students' needs. 

• Reflected on the experience of teaching the unit with an ASHP History 
Educator; selected and annotated student work to identify student 
struggles and strengths. 

• Revised lessons and materials based on reflection. 

• Why? Teaching model lessons, then reflecting on the resulting classroom 
experience and student work with expert mentors, gives teachers the 
guidance and practice necessary to design their own Common Core 
aligned history curriculum. 

• Earning this badge allows the teacher to proceed to Journeyman Builder. 

• Work for this badge represents 8-10 professional development hours. 

The Journeyman Builder has continued on the path toward becoming a Master 
History Teacher who creates effective Common Core aligned lessons in U.S. 
history. To earn the Journeyman Builder badge, the teacher has: 

• Studied the [unit title] Who Built America inquiry unit and passed a 
multiple-choice quiz with a score of [quiz score] to demonstrate 
understanding of its historical content and Common Core standards. 

• Prepared to teach the inquiry unit by revising lessons and materials to 
meet students’ needs. 

• Reflected on the experience of teaching the unit with an ASHP History 
Educator; selected and annotated student work to identify student 
struggles and strengths. 

• Revised lessons and materials based on reflection. 

• Why? Teaching model lessons, then reflecting on the resulting classroom 
experience and student work with expert mentors, gives teachers the 
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guidance and practice necessary to design their own Common Core 
aligned history curriculum. 

• Earning this badge allows the teacher to proceed to Master Builder. 

• Work for this badge represents 8-10 professional development hours. 

The Master Builder has reached the final stage on the path toward becoming a 
Master History Teacher who creates effective Common Core aligned lessons in 
U.S. history. To earn the Master Builder badge, the teacher has: 

• Planned a unit by choosing one ofWBA’s essential questions 

• Created an inquiry unit 

• Reflected on the experience of teaching the unit with an ASHP History 
Educator 

• Selected and annotated student work to identify student struggles and 
strengths. 

• Revised lessons and materials based on reflection. 

• Work for this badge represents 8-10 professional development hours. 
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APPENDIX B: TEACHER PROFILE PAGE SCREENSHOT 



JIM DIAMOND 


OVERVIEW ALL SUBMISSIONS MANAGE DISPLAY MY SUBMISSIONS MY SETTINGS 

TRACK PAGE VISITS COMMUNITY SUBSCRIPTIONS FILE BROWSER DEVEL Start the tourl 



Center for Children & Technology 
New York 


SUBMISSION CHAT 


You don't currently have any 
form submissions to review, 
but after you've created a 
submission toward earning a 
badge, the latest feedback 
from ASHP staff will appear 
here. 


A, MY BUILDER BADGES 



Dawn of Empire: 
The U.S. War in the 
Philippines 



O 

o 

You passed 
the quiz. - 
showing that 
you know your 
stuff. 





RECENT COMMUNITY 


Developing Common Core 
Lessons 


10 posts, most recently from 
|jm^^>n 6/27 at 
12:29am: Accessible history 


Materials For 
Implementation 

y posts, most recently from 
5/16 at 

1 :1 0pm: Education Portal 

Read More » 


£* MY COMMUNITY BADGES 



WBA Member 

Create a WBA 
Badges Account 



I Love Community 


Start topics, reply 
to posts, mark 
what's useful 



Constructive Critic 

Add to the 
conversation in 
ways that others 
find useful 



I Love Sharing 

Create 3 sharing 
posts 


Elementary Instruction 

4 posts, most recently from 
4/28 at 

8:00am: nonimportation 


Read More » 


O MY SPECIALIST BADGES 



Common Core 
Reading Specialist 

Upload your best 
Common Core 
Reading Lesson 



Common Core 
Writing Specialist 


Upload your best 
Common Core 
Writing Lesson 



History Geek 

Pass 4 Unit 
Quizzes 


Tech Guru 

Upload your best 
technology or 
media lesson 


MASTER HISTORY TEACHER 



Master History Teacher 


Be a Master Builder, Community Member, and Specialist. 
Become a Master History Teacher by demonstrating: 

\ Effective Common Core teaching through reflection, 
analysis of student work, and revision. 

SZ* Active membership in a professional community where 
you share knowledge and resources. 

Q Expertise in key aspects of history education— the 
Common Core, historical content, and using technology. 
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APPENDIX C: SUBMISSION FORM EXAMPLE 
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APPENDIX D: INTERVIEW PROTOCOL 

1. What initially interested you about the Who Built America web site? (Probes: 
Were you specifically interested in professional development? The badges? 
Materials? Feedback on lessons? The study stipend?) 

2 . (If s/he comments on a specific interest in professional development): What 
kind of professional development were you interested in? 

3. About how many times did you visit the WBA site? (Probe: Encourage them to 
be honest. Why did you visit the site? What did you look at? What were your 
impressions?) 

4 . Did you look at the digital badges on the site? What did you think about them? 
(Probes: Had you heard about digital badges before? Did they make sense in 
terms of being professional development credentials?) 

5. What are your recollections about the professional development format? 

(Probe: Was it appealing? Why or why not?) 

6. About when did you realize that you probably wouldn't continue using the site? 
(Probe: Were you busy with work or otherwise? Were their features or 
functionalities about the site that made it less useful to you for professional 
development?) 

7. What would make the site better, given your professional development needs? 

8. Did you discuss the site with anyone else? (Probe: Who? What did you discuss?) 

9. About how often do you engage in professional development activities? (Probe: 
Why do you engage in professional development activities? What are better 
formats? What are worse formats?) 

10 . Do you receive recognition when you go through professional development? 
(Probe: If so, in what ways? Who recognizes it? Is it important to you that 
others recognize it?) 

11. In your experience, do teachers and administrators make judgments about 
each other’s skills based on their credentials? (Probe: By credentials, I mean 
college degree or professional development experience 

12 . Is there anything else that you’d like to discuss about Who Built America? or 
professional development in general? 


